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Abstract 

Background: Health care news stories have the potential to inform and educate news consumers and health-care 
consumers about the tradeoffs involved in health-care decisions about treatments, tests, products, and procedures. 
These stories have the potential to influence not only individual decision making but also the broader public 
dialogue about health-care reform. For the past 7 years, a Web-based project called http://HealthNewsReview.org 
has evaluated news stories to try to improve health-care journalism and the quality and flow of information to 
consumers. 

Analysis: http://HealthNewsReview.org applies 10 standardized criteria to the review of news stories that include 
claims about medical interventions. Two or three reviewers evaluate each story. The team has evaluated more than 
1,800 stories by more than a dozen leading U.S. news organizations. About 70% have received unsatisfactory scores 
based on application of these criteria: reporting on costs, quantifying potential benefits, and quantifying potential 
harms. 

Conclusions: Inaccurate, imbalanced, incomplete news stories may drown out more careful scrutiny of the 
evidence by many influential news organizations. Unquestioned claims and assertions about the benefits of 
medical interventions are passed on to the American public daily by journalists who are supposed to vet 
independently any such claims. Communication about these issues is, in itself a major health-policy issue. 



Background 

At a time when many Americans may be hearing about 
comparative effectiveness research for the first time, many 
health care news stories by U.S. news media emphasize or 
exaggerate the potential benefits of interventions while 
minimizing or ignoring the potential harms. Ironically, as 
the health-care reform bill in the United States was named 
the Patient Protection and Affordable Care Act (Public 
Law 11-148), most of the news stories released by the U.S. 
news media about medical interventions failed to ade- 
quately discuss cost — or affordability — issues. 

In an era when many health insurance companies are 
marketing high-deductible or so-called consumer-driven 
health-care plans predicated on the assumption that 
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consumers will be more cost-conscious if they receive 
more information about the cost of care and quality of 
their providers, much of the news and information they 
receive every day through mass media messages doesn't 
help them define or assess such quality. While evidence- 
based medicine proponents promote a shared decision- 
making discussion of the tradeoffs between harms and 
benefits in screening test decisions, the public dialogue 
may be overtaken by proscreening advocates who treat 
screening as a mandate, not a choice. 

Deyo and Patrick wrote, "Vested interests, marketing, 
politics and media hype often have more influence on how 
new medical advances get used than the best scientific evi- 
dence" [1]. The intersection of medicine and the media is 
often a messy place, and the public may be harmed as a 
result. 
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This paper will document recurring problems in media 
messages about health-care interventions and will offer 
suggestions for new or expanded efforts to improve the 
public dialogue about health care. 

Analysis 

This paper will review the findings from http://Health- 
NewsReview.org, a unique project that evaluates and tries 
to improve the quality of health-care media messages. It 
will also provide a closer look at some specific problems in 
health-care journalism, including some that were observed 
in news coverage of the revised mammography recom- 
mendations released by the U.S. Preventive Services Task 
Force (USPSTF) in 2009. 

For 7 years, the http://HealthNewsReview.org project 
has evaluated news stories about healthcare interventions 
(treatments, tests, products, procedures) by leading news 
organizations in the United States. Ten standardized cri- 
teria are applied to the review of each story. These cri- 
teria represent journalism issues identified either in a 
New England Journal of Medicine paper in 2000 [2] or in 
the Association of Health Care Journalists "Statement of 
Principles" [3]. 

The criteria for analysis include these questions: 

♦ Does the story adequately discuss the costs of the 
intervention? 

♦ Does the story adequately quantify the benefits of the 
treatment/test/product/procedure? (A story that frames 
benefits only in relative [not absolute] risk reduction 
terms is likely to get an unsatisfactory score.) 

♦ Does the story adequately explain/quantify the 
harms of the intervention? 

♦ Does the story seem to grasp the quality of the 
evidence? 

♦ Does the story commit disease-mongering? 

♦ Does the story use independent sources and iden- 
tify conflicts of interest in sources? 

♦ Does the story compare the new approach with 
existing alternatives? 

♦ Does the story establish the availability of the 
treatment/ test/ product/procedure? 

♦ Does the story establish the true novelty of the 
approach? 

♦ Does the story appear to rely solely or largely on a 
news release? 

Data from the http://HealthNewsReview.org project 
appear in Table 1. 

Each story is reviewed by two to three reviewers from a 
team of people trained to evaluate medical evidence [4] . 
Journalists are notified by email whenever one of their 
stories is reviewed. 



The daily drumbeat of imbalanced messages 

About two out of every three news stories reviewed fail to 
adequately address costs and fail to adequately quantify 
harms and benefits. For an American audience that may 
not even be aware that its nation spends a world-leading 
17% of its gross domestic product on health care [5], 
leaving more than 16% of the population uninsured [6], 
journalists may not be delivering vital information on 
important public policy issues. The dissonance between 
comparative effectiveness research and daily media mes- 
sages that emphasize or exaggerate the benefits of inter- 
ventions and minimize or ignore their harms is striking. 
Recurring themes in these media misadventures include: 

• Stories may frame benefits in the most positive light 
by using relative risk reduction statistics without the 
corresponding absolute risk reduction numbers. 

« The tyranny of the anecdote: Stories may use only 
positive, glowing patient anecdotes but fail to mention 
trial dropouts, compliance problems, patient dissatis- 
faction, or the choice to pursue less aggressive options. 

♦ Stories may fail to explain the limitations of observa- 
tional studies and conflate association with causation. 
Misreporting of observational studies may lead readers 
to question the credibility of both journalism and 
science. 

« Stories may frame surrogate markers or intermedi- 
ate endpoints as if they were outcomes on which 
consumers should focus. 

« Many news organizations seem to believe that any- 
thing published in a medical journal is infallible and 
unquestionably newsworthy. Journalists should 
become familiar with the work of Stanford University 
professor John loannidis, such as "Why Most Pub- 
lished Research Findings Are False" [7]. Increasingly, 
many news organizations cover more stories out of 
journals each week — often as a cost-savings measure 
because reporters can cover these stories without leav- 
ing the newsroom. But journalists who feed off of a 
steady diet of studies published in journals may not be 
aware of the publication bias in favor of positive find- 
ings in many journals. So that steady diet of journal 
stories may convey to readers an overly optimistic 
view of research progress. 

« About half of all stories reviewed are single-source 
stories and/or those that fail to disclose conflicts of 
interest in sources. This approach should not give 
readers confidence in the balance and integrity of the 
story. 

An uninformed public dialogue about screening 

What was seen in much (not all) news coverage of the 
November 2009 recommendations of the USPSTF on 
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Table 1 Results from review of 1,854 health news stories by HealthNewsReview.org 

Criterion No. of satisfactory No. of unsatisfactory No. of not applicable (N/A) Percentage of satisfactory scores (total 











mini ic N / 
iiiiiius I'll n) 


Costs 


4/0 


1 nQ7 
,Uo/ 


zy 1 




Benefits 


623 


1,204 


27 


34% 


Harms 


626 


1,158 


70 


35% 


Evidence 


713 


1,137 


4 


39% 


Disease-mongering 


1,385 


397 


72 


78% 


Sources/conflict of 


1,021 


829 


4 


55% 


interest 










Alternatives 


781 


1,021 


52 


43% 


Availability 


1,227 


450 


177 


76% 


Novelty 


1,372 


381 


101 


78% 


Rely on press 


1,329 


120 


405 


92% 


release* 











* The criterion asking whether the story appeared to rely solely or largely on a news release requires some explanation. Note the high number of not applicable 
(N/A) scores. In order to make a judgment on this criterion, reviewers must have a copy of a news release. Because a news release is not always available, many 
stories are graded N/A. The percentage of satisfactory scores may seem high; another perspective is that the fact that 120 supposedly independently reported 
stories were found to rely solely or largely on a news release is troubling. 



screening for breast cancer was disturbing because of the 
imbalanced coverage in many news stories. Many stories 
demonstrated a lack of Icnowledge about the USPSTF, a 
lack of appreciation for the expertise of its members, and 
failure to grasp the fact that screening tests may carry dif- 
ferent harms and benefits for different age groups. The 
framing of many stories was built on themes of a govern- 
ment task force trying to save money or ration care, of 
government deciding that "some lives don't matter," or of 
personal anecdotes of women who claimed their lives had 
been saved by mammograms. In the patient or consumer 
interviews some stories chose to highlight, people said 
they were shocked by the recommendations because they 
"seemed to come out of nowhere" or "out of the blue." 
Many stories did little to even hint at the long historical 
context of the ongoing uncertainty over the benefits and 
harms of breast cancer screening, especially in younger 
women. Finally, some stories were framed around mes- 
sages that said the "shifting recommendations prove that 
scientists are clueless"; others questioned the expertise of 
USPSTF members because none had subspecialty training 
in oncology or radiology. 

The important shared decision-making message of the 
recommendations published by the USPSTF never 
appeared in many stories, columns or talk shows that fol- 
lowed. The USPSTF wrote: "The decision to start regular, 
biennial screening mammography before the age of 50 
years should be an individual one and take patient context 
into account, including the patient's values regarding spe- 
cific benefits and harms." That message was either missed 
or ignored in many stories and became a topic of ridicule 
in other instances — almost as if uncertainty was something 
to be shunned at all costs. 

The editors of the Annals of Internal Medicine published 
an editorial, "When Evidence Collides With Anecdote, 



Politics, and Emotion: Breast Cancer Screening" [8], in 
which they referred to a "media cacophony": 

"...[T]he media and politicians presented the breast 
cancer screening recommendations as a major depar- 
ture from existing guidelines that heralded an age of 
rationed care in the United States. Confusion, politics, 
conflicted experts, anecdote, and emotion ruled front 
pages, airwaves, the Internet, and dinner-table conver- 
sations.... 

The initial reaction to the Task Force recommenda- 
tions might have been less vehement had the potential 
negative consequences of alternative recommendations 
also been considered." 

The true potential harms of screening were rarely com- 
municated. Veteran science journalist John Crewdson 
wrote in The Atlantic [9]: 

"The current controversy over the task force's report 
owes much to the media's confusing coverage, some of 
which has been misinformed, including by TV doctors 
who ought to know better. 

...There are multiple reasons women are ill-informed 
about breast cancer. The fault lies primarily with their 
physicians, the cancer establishment, and the news 
media — especially the news media. Until coverage of 
breast cancer rises above the level of scary warnings 
mixed with heartwarming stories of cancer survivors, 
women are likely to go on being perplexed." 

The challenge for those communicating evidence-based 
health-care messages is clearly shown in this one episode 
from our recent history. But it has been repeated with 
news coverage of evidence-based statements on prostate 
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cancer screening, lung cancer screening, and coronary 
artery disease screening. 

Public confusion 

The Kaiser Family Foundation Health Tracking Poll shows 
that Americans are confused about much of what they 
hear and read about health care [10]. In a May 2010 poll, 
the Kaiser Family Foundation found that 44% of those 
polled were confused about the health-care reform legisla- 
tion. Journalists play a role in creating that confusion. 

Of Americans polled, 63% got health-care information 
from cable TV news programs and 55% from broadcast 
news programs. From the Kaiser Family Foundation report: 
"Further brealcing down those getting health reform infor- 
mation from cable news, 25 percent of Americans indi- 
cated their main cable source on this topic was FOX News, 
22 percent named CNN and 6 percent MSNBC. In fact, 
cable news still tops the list of the public's 'most important' 
sources of news about the new law, with 30 percent saying 
they rely on that source more than any other." 

Public confusion is fueled, not helped, by journalists who 
promote unquestioned claims and assertions with a com- 
bination of naivete, anecdotes, emotions, and editorializing. 

Conclusions 

A call to action could arise from a recent study published 
in Health Affairs, "Evidence That Consumers Are Skeptical 
About Evidence-Based Health Care" [11]. Its findings were 
chilling: 

"...[0]ur findings illuminate real and significant chal- 
lenges to the pursuit of broader acceptance of evi- 
dence-based health care among consumers. The 
beliefs underlying the themes that surfaced in both the 
qualitative research and the survey — more is better, 
newer is better, you get what you pay for, guidelines 
limit my doctor's ability to provide me with the care I 
need and deserve — are deeply rooted and widespread. 
Our findings, although preliminary, have implications 
for several public and private efforts. These efforts 
intend to foster — and to some degree depend on — 
consumer engagement, or at least on the absence of 
overt consumer resistance." 

Many of the concepts that need to be communicated to 
the public under the umbrella of evidence-based guide- 
line-setting are counterintuitive. The fact that they are 
counterintuitive should be addressed early, often, and 
clearly. Decades of "first impressions" from other sources 
have molded the thinking of many people. For those who 
were raised being told they should fight cancer "with a 
checkup and a check," it may be very difficult to grasp 
that what is often framed as "a simple blood (or other 
screening) test" may actually carry potential harms as 



well as benefits. Communicators need to learn how to 
communicate about the tradeoffs of harms and benefits 
in screening tests and treatment decisions and about 
shared decision making that allows the individual to 
address his/her own values in the face of uncertainties. 

Some steps that could be taken 

Some excellent journalism training opportunities exist. 
Others have been shut down or been threatened by lack of 
funding. The following is an outline of activities that fun- 
ders could consider. 

• Government-funded efforts 

After the November 2009 mammography screening epi- 
sode, it was clear that many journalists and many citizens 
have little knowledge about the USPSTF, its mission, or 
its members. The Agency for Healthcare Research and 
Quality (AHRQ), a U.S. federal government research 
agency, could launch an educational outreach effort 
about the work of the USPSTF. It would seem that the 
Patient-Centered Outcomes Research Institute — "author- 
ized by [the U.S.] Congress to conduct research to pro- 
vide information about the best available evidence to 
help patients and their health care providers make more 
informed decisions" — would also want to invest in educa- 
tional efforts for journalists and the general public. 

The National Institutes of Health have been very success- 
ful in training journalists in the United States how to scru- 
tinize medical evidence in their annual media workshops 
[12]. (Note: The 2013 Medicine in the Media course was 
cancelled due to federal budget sequestration.) For 14 
years, the Rocky Mountain Workshop on How to Practice 
Evidence-Based Health Care, with AHRQ support, has 
often invited journalists to learn about medical evidence 
and share ideas and challenges with physicians, public 
health professionals, and policymakers [13]. 

• Foundation efforts 

The Knight Foundation supports an annual Medical 
Evidence Boot Camp at the Massachusetts Institute of 
Technology (MIT). The California Endowment funds 
training, Web publishing and fellowships at the University 
of Southern California's Annenberg School for Communi- 
cation and Journalism. The investigative journalism of 
ProPublica's newsroom (http://www.propublica.org) — 
much of it on health care issues — is supported by the 
Sandler Foundation, the Knight Foundation, the 
MacArthur Foundation, the Pew Charitable Trusts, the 
Ford Foundation, and the Carnegie Corporation. 

Foundations could do more. It seems that for every spe- 
cialized science/medical/health journalism graduate 
program that has survived (e.g., University of Georgia, 
University of North Carolina-Chapel Hill, New York Uni- 
versity, Boston University, University of California-Santa 
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Cruz, Columbia University, MIT), others are shut down 
for apparent economic reasons (the University of Minne- 
sota master's program in health journalism, the Johns 
Hopkins University master's program in science writing, 
and the Florida Atlantic University science writing pro- 
gram). Foundations could help support such efforts. 

http://HealthNewsReview.org was supported by the 
Informed Medical Decisions Foundation for 8 years, but 
that funding ends in 2013 with no replacement funding 
in place at this time. Its German counterpart, Medien- 
Doktor (http://www.medien-doktor.de/english/) benefits 
from foundation support. But the pioneering effort of 
this genre. Media Doctor Australia (http://www.media- 
doctor.org.au/), stopped publishing in 2012 because of 
lack of funding, the same fate that befell a Media Doctor 
Canada project earlier. Foundations could help these 
efforts as well. 

• Not-for-profit organization layperson training 
effort 

Another model could be the popular Project LEAD 
science training course offered by the National Breast 
Cancer Coalition (NBCC) to its members [14]. Project 
LEAD courses prepare attendees to understand the basics 
of breast cancer science, how the biomedical research pro- 
cess works, and how to more accurately understand scien- 
tific information in the news. 

At an Institute of Medicine workshop, Fran Visco, presi- 
dent of NBCC, criticized the USPSTF's communication 
and dissemination strategies outlined in a 2007 report [15]. 
She said: 

"Unfortunately, these strategies focused on dissemina- 
tion to professional audiences via medical journals and 
federal agencies, professional societies, and quality 
improvement organizations via public meetings as the 
primary audiences. As the recent breast cancer screen- 
ing example illustrates, strategies for disseminating 
information to the media, poUcy makers, and the pub- 
lic are also crucial components of any communication 
plan." 

What if? What if not? 

If evidence-based medicine advocates can make progress 
in any of the areas described above, perhaps they will gain 
a foothold in advancing meaningful health-care reform. 

If these advocates fail to make progress, the authors of a 
recent Health Affairs paper provide a somber warning: 

"To the extent that consumers perceive that the appli- 
cation of comparative effectiveness research to decision 
making could limit their choice of providers, inappro- 
priately interfere with physicians' recommendations for 



treatment, or appear to "ration" care based on cost, 
these efforts will encounter consumer resistance and 
could lead to a broad consumer backlash.... 
Effective communication with and support of consu- 
mers is essential to improving the quality of health care 
and containing health care costs. Clearly, consumers 
will revolt if evidence-based efforts are perceived as 
rationing or as a way to deny them needed treatment. 
Policy makers, employers, health plans, providers, and 
researchers will thus need to translate evidence-based 
health care into accessible concepts and concrete activ- 
ities that support and motivate consumers. A necessary 
condition for effective communication, after all, is to 
start where your audience is — even if that is not where 
you hoped or expected it to be" [11]. 

It is clear that many U.S. citizens are not where evi- 
dence-based health-care proponents wish they would be. 
And it's also clear that many who balk at these principles 
gravitate toward the benefits of interventions that have 
been so successfully drummed into them and are deaf to 
the potential harms and/or absence of benefit that have 
been underemphasized or ignored entirely in many forms 
of communication. 

Independent of the delivery of care, communication 
about these issues to and with the American public — by 
government agencies, providers, researchers, medical jour- 
nal editorial boards, advocacy groups, the drug and device 
industry, and journalists — is, itself, a major health-policy 
issue. 

Abbreviations used 

MIT: Massachusetts Institute of Technology; NBCC: National Breast Cancer 
Coalition; USPSTF: U.S. Preventive Services Task Force 

Competing interests 

None. 

Acl<nowledgements 

The http://HealthNewsReview.org project was supported by the Informed 
Medical Decisions Foundation of Boston, MA, from August 2005 through 
July 2013. 

Declarations and disclaimer 

The Eisenberg Conference Series 2012, Supporting Informed Decision IVSaking 
When Clinical Evidence and Conventional Wisdom Collide, was conducted in 
Rockville, Maryland, by the John M. Eisenberg Center for Clinical Decisions 
and Communications Science, Baylor College of Medicine, Houston, Texas, 
for the Agency for Healthcare Research and Quality under Contract No. 
HHSA 290-2008-1 001 5-C. Publication costs for this supplement were funded 
by this contract. The author of this article is responsible for its content No 
statement may be construed as the official position of the Agency for 
Healthcare Research and Quality and of the U.S. Department of Health and 
Human Services. 

This article has been published as part of BMC Medical Informatics and 
Decision Maldng Volume 13 Supplement 3, 2013: Articles from the Eisenberg 
Center Conference Series 2012: Supporting informed decision making when 
clinical evidence and conventional wisdom collide. The full contents of the 
supplement are available online at http://www.biomedcentraLcom/ 
bmcmedinformdecismak/supplements/1 3/S3. 



Schwitzer BMC Medical Informatics and Decision Making 2013, 13(Suppl 3):S3 
http://www.biomedcentral.eom/1 472-6947/1 3/S3/S3 



Page 6 of 6 



Authors' details 

'HealthNewsReview.org, Saint Paul, MN 551 16, USA. ^University of Minnesota 
School of Public Health, IVlinneapolis, Minnesota, 55455, USA. 



Published: 6 December 2013 



References 

1. Deyo RA, Patrick Dt: Hope or Hype: The Obsession With Medical 
Advances and the High Cost of False Promises. New York: American 
Management Association; 2005. 

2. Moynihan R, Bero t, Ross-Degnan D, Henry D, Lee K, Watkins J, Mah C, 
Soumerai SB: Coverage by the news media of the benefits and risks of 
medications. N Engl J Med 2000, 342:1645-1650. 

3. Schwitzer G: A statement of principles for health care journalists. Am 
J Bioeth 2004, 4:W9-W13. 

4. HealthNewsReview.org: Reviewers, [httpy/www.healthnewsreview.org/ 
about-us/reviewers/j. 

5. CMS.gov. Centers for Medicare and Medicaid Services: National Health 
Expenditure DATA: NHE Fact Sheet. [https://www.cms.gov/Research- 
Statistics-Data-and-Systems/Statistics-Trends-and-Reports/ 
NationalHealthExpendData/NHE-Fact-Sheet.html]. 

6. Assistant Secretary for Planning and Evaluation, U.S. Department of Health 
and Human Services: ASPE Issue Brief: Overview of the Uninsured in the 
United States: A Summary of the 201 1 Current Population Survey. 
[httpy/aspe.hhs.gov/health/reports/2011/CPSHealthlns2011/ib.pdf|. 

/. loannidis JPA: Why most published research findings are false. PLoS Med 
2005, 2:el24 

8. The Editors: When evidence collides with anecdote, politics, and 
emotion: breast cancer screening. Ann Intern Med 2010, 152:531-532. 

9. Crewdson J: Rethinking the mammogram guidelines. The Atlantic 2009 
[httpy/wwwtheatlantic.com/magazine/archive/2009/1 1/rethinking-the- 
mammogram-guidelines/7791/], 19 November. 

10. Henry J, Kaiser Family Foundation: Kaiser health tracking poll — May 2010. 
[httpy/www.kff.org/kaiserpolls/8075.cfm]. 

11. Carman KL, Maurer M, Yegian JM, Dardess P, McGee J, Evers M, Mario KO: 
Evidence that consumers are skeptical about evidence-based health 
care. Health Aff (Millwood) 2010, 29:1400-1406. 

12. National Institutes of Health Office of Disease Prevention: Medicine in the 
media: reporting on medical research: Medicine in the Media course, 
[httpy/medmediacourse. nih.gov/]. 

13. Colorado School of Public Health: Evidence-based health care: the Rocky 
Mountain Workshop on How to Practice Evidence-Based Health Care. 
[httpy/www.ucdenver.edu/academics/colleges/PublicHealth/community/ 
ebhc/Pages/default.aspx]. 

14. National Breast Cancer Coalition: Project LEAD. [http://www. 
breastcancerdeadline2020.org/get-involved/training/project-lead/]. 

15. Guirguis-Blake J, Calonge E, Miller T, Siu A, Teutsch S, Whitlock E: Current 
processes of the U.S. Preventive Services Task Force: refining evidence- 
based recommendation development. Ann Intern Med 2007, 147:1 17-122. 

' • 

doi:1 0.1 1 86/1 472-6947-1 3-S3-S3 

Cite this article as: Schwitzer: Addressing tensions when popular media 
and evidence-based care collide. BMC Medical Informatics and Decision 
Making 2013 13(Suppl 3):S3. 



Submit your next manuscript to BiolVled Central 
and tal<e full advantage of: 

• Convenient online submission 

• Thorough peer review 

• No space constraints or color figure charges 

• Immediate publication on acceptance 

• Inclusion in PublVled, CAS, Scopus and Google Scholar 

• Research which is freely available for redistribution 



Submit your manuscript at A Tpntral 

www.biomedcentral.com/submit \ J BKMViea central 



